热门标签 | HotTags
当前位置:  开发笔记 > 编程语言 > 正文

Pipeinfor循环中断了双引号变量-Pipeinforloopbreaksdoublequotedvariables

Situation:UsingbatchscripttoretrievecertainvaluesfromaJSON.Ivegotthefollowingbatchscr

Situation: Using batchscript to retrieve certain values from a JSON.
I've got the following batchscript:

情况:使用batchscript从JSON中检索某些值。我有以下批量脚本:

@ECHO off

ECHO Enter npo.nl program-url :
SET url=
SET /P url=
:: http://www.npo.nl/buitenhof/03-05-2015/VPWON_1232766/POMS_VPRO_850040 for example
SETLOCAL ENABLEDELAYEDEXPANSION
FOR /F "tokens=6 delims=/" %%A IN ("%url%") DO (
    FOR /F "delims=" %%B IN ('curl.exe -s http://e.omroep.nl/metadata/aflevering/%%A ^| jq.exe -R -r -s ".[1+index(\"^(\"): rindex(\"^)\")]"') DO (
        FOR /F "delims=" %%C IN ('ECHO %%B ^| jq.exe -r .start') DO SET ss=%%C
        FOR /F "delims=" %%C IN ('ECHO %%B ^| jq.exe -r .eind') DO SET to=%%C
        FOR /F "delims=" %%C IN ('ECHO %%B ^| jq.exe -r .tijdsduur') DO SET t=%%C
        ECHO Start:    !ss!
        ECHO End:      !to!
        ECHO Duration: !t!
    )
)
ENDLOCAL
PAUSE

What does it do?
Upon having entered a npo.nl program-url, the first for-loop strips the url down to the prid:POMS_VPRO_850040. In the second for-loop curl.exe retrieves the JSON...:

它有什么作用?在输入npo.nl program-url后,第一个for-loop将url删除到prid:POMS_VPRO_850040。在第二个for循环中curl.exe检索JSON ...:

parseMetadata({"STATUS":"OK","VERSION":"1.11.12","prid":"VPWON_1232766","titel":"Schuim & As","aflevering_titel":"","info":"Schuim & As met Jelle Brandt Corstius","ratio":"16:9","medium":"tv","gidsdatum":"2015-05-03","tijdsduur":"00:05:27","start":"00:23:13","eind":"00:28:40","url":"","webcast":1,"images":[{"size":"640x480","ratio":"4:3","url":"http:\/\/images.poms.omroep.nl\/image\/sx480\/c640x480\/606030.jpg"},{"size":"720x405","ratio":"16:9","url":"http:\/\/images.poms.omroep.nl\/image\/sx405\/c720x405\/606030.jpg"}],"omroepen":[{"naam":"VPRO"}],"pubopties":["adaptive","h264_bb","h264_sb","h264_std"],"tt888":"ja","serie":{"srid":"VPWON_1232748","serie_titel":"Buitenhof","serie_url":null},"sitestat":{"baseurl":"http:\/\/b.scorecardresearch.com\/p?c1=2&c2=17827132&ns_site=po-totaal","programurl":"uitzendinggemist.publiekeomroep.ondemand.tv.buitenhof.20150503","programurlpost":"category=uitzendinggemist&thema=informatief&po_source=video","baseurl_subtitle":"http:\/\/nl.sitestat.com\/klo\/po\/s","subtitleurl":"uitzendinggemist.publiekeomroep.ondemand.tv.player.tt888.buitenhof","subtitleurlpost":"category=uitzendinggemist&po_source=video&po_sitetype=webonly"},"reclame":"http:\/\/pubads.g.doubleclick.net\/gampad\/ads?_COOKIE_&impl=s&gdfp_req=1&env=vp&output=xml_vast2&unviewed_position_start=1&sz=_sz_&correlator=_correlator_&iu=\/9233\/_site_\/buitenhof&url=_url_&cust_params=genre%3Dinformatief%2Cnieuws%2Factualiteiten%26dur%3D3284%26prid%3DVPWON_1232766%26srid%3DVPWON_1232748%26player%3D_player_","streamSense":{"episode":"buitenhof","program":"buitenhof","station":"nederland_1","sitestatname":"uitzendinggemist.publiekeomroep.ondemand.tv.buitenhof.20150503","sko":"TRUE","sko_dt":"20150503","sko_pr":"buitenhof","sko_stid":"1","sko_ty":"tv.seg","sko_prid":"vpwon1232766","sko_t":"1210","sko_cl":"3284"}})
//epc

...and sends it through a pipe to jq.exe which removes the non-JSON-data parseMetadata( and ) //epc and leaves the single line intact. This is for 2 reasons: 1) with non-JSON-data present we can't process anything, and 2) for-loops process only 1 line at a time.
The subsequent jq.exe's retrieve values for the specified objects without double quotes.
As long as curl.exe and jq.exe are in the same directory as the batchscript, or in the %path%-variable, this is working all fine:

...并通过管道将其发送到jq.exe,这将删除非JSON数据parseMetadata(和)// epc并使单行保持不变。这有两个原因:1)当存在非JSON数据时,我们无法处理任何内容,2)for循环一次只处理1行。后续jq.exe检索指定对象的值,不带双引号。只要curl.exe和jq.exe与batchscript位于同一目录中,或者在%path%-variable中,这样就可以了:

Start:     00:23:13
End:       00:28:40
Duration:  00:05:27

Now I want to call curl.exe and jq.exe from another map. One with spaces in it:

现在我想从另一个地图调用curl.exe和jq.exe。其中一个有空格:

SET curl="C:\map with spaces\curl.exe"
SET jq="C:\map with spaces\jq.exe"

@ECHO off

ECHO Enter npo.nl program-url :
SET url=
SET /P url=
:: http://www.npo.nl/buitenhof/03-05-2015/VPWON_1232766/POMS_VPRO_850040 for example
SETLOCAL ENABLEDELAYEDEXPANSION
FOR /F "tokens=6 delims=/" %%A IN ("%url%") DO (
    FOR /F "delims=" %%B IN ('%curl% -s http://e.omroep.nl/metadata/aflevering/%%A ^| %jq% -R -r -s ".[1+index(\"^(\"): rindex(\"^)\")]"') DO (
        FOR /F "delims=" %%C IN ('ECHO %%B ^| %jq% -r .start') DO SET ss=%%C
        FOR /F "delims=" %%C IN ('ECHO %%B ^| %jq% -r .eind') DO SET to=%%C
        FOR /F "delims=" %%C IN ('ECHO %%B ^| %jq% -r .tijdsduur') DO SET t=%%C
        ECHO Start:    !ss!
        ECHO End:      !to!
        ECHO Duration: !t!
    )
)
ENDLOCAL
PAUSE

For the 2nd for-loop this causes problems:

对于第二个for循环,这会导致问题:

'C:\map' is not recognized as an internal or external command,
operable program or batch file.

While 'ECHO %%X ^| %jq%' does work, it seems '%curl% ^| %jq%' doesn't. So for some reason things go wrong as soon as 2 variables in a pipe are parsed.

而'ECHO %% X ^ | %jq%'确实有效,似乎'%curl%^ | %jq%'没有。因此,出于某种原因,只要解析了管道中的2个变量,就会出现问题。

Well, no more pipe then:

那么,没有更多的管道:

SET curl="C:\map with spaces\curl.exe"
SET jq="C:\map with spaces\jq.exe"

@ECHO off

ECHO Enter npo.nl program-url :
SET url=
SET /P url=
:: http://www.npo.nl/buitenhof/03-05-2015/VPWON_1232766/POMS_VPRO_850040 for example
SETLOCAL ENABLEDELAYEDEXPANSION
FOR /F "tokens=6 delims=/" %%A IN ("%url%") DO (
    FOR /F "delims=" %%B IN ('%curl% -s http://e.omroep.nl/metadata/aflevering/%%A') DO (
        FOR /F "delims=" %%C IN ('ECHO %%B ^| %jq% -R -r -s ".[1+index(\"^(\"): rindex(\"^)\")]"') DO (
            FOR /F "delims=" %%D IN ('ECHO %%C ^| %jq% -r .start') DO SET ss=%%D
            FOR /F "delims=" %%D IN ('ECHO %%C ^| %jq% -r .eind') DO SET to=%%D
            FOR /F "delims=" %%D IN ('ECHO %%C ^| %jq% -r .tijdsduur') DO SET t=%%D
            ECHO Start:    !ss!
            ECHO End:      !to!
            ECHO Duration: !t!
        )
    )
)
ENDLOCAL
PAUSE

Now curl.exe and jq.exe each in a for-loop. At first this seems to work fine. The 3 values are echoed, but then things go wrong:

现在curl.exe和jq.exe都在for循环中。起初这似乎工作正常。回显了3个值,但事情出错了:

parse error: Invalid numeric literal at line 1, column 5

parse error: Invalid numeric literal at line 1, column 5
parse error: Invalid numeric literal at line 1, column 5
parse error: Invalid numeric literal at line 1, column 5
Start:    00:23:13
End:      00:28:40
Duration: 00:05:27

Like I said before; for-loops parse and process only 1 line at a time. The non-JSON-data //epc on the 2nd line causes the for-loop to start over, which goes horribly wrong as you can see. That's the reason for the pipe between curl and jq in the code above. To output 1 single line to process. Sadly that didn't work either...sigh.

就像我之前说的那样; for-loops一次只解析和处理1行。第二行上的非JSON数据// epc导致for循环重新开始,如你所见,这非常错误。这就是上面代码中curl和jq之间管道的原因。输出1个单行进行处理。可悲的是,这也无济于事......叹息。

Of course using temporary files is a last resort when curl and jq are still in a map with spaces in it, but I prefer to use variables, so I'm trying to solve the pipe-issue. I've tried 'usebackq' in the for-loop using backticks around the command instead of single-quotes for instance, but to no avail.
So far I haven't found a solution. Does anyone have an explanation for this behaviour and how to solve it?

当然,当curl和jq仍在带有空格的地图中时,使用临时文件是最后的手段,但我更喜欢使用变量,所以我正在尝试解决管道问题。我在for循环中尝试了'usebackq',而不是使用单引号的反引号,但是无济于事。到目前为止,我还没有找到解决方案。有没有人对此行为有解释以及如何解决?

2 个解决方案

#1


Thanks to Dave Benham's answer on a related issue I've found the solution!
It appeared to be a specific FOR /F bug in WinXP, and guess what, here I'm still on WinXP. To fix the main offender, the curl-pipe-jq-for-loop, I had to put ^" in front of and after the entire command. Thus the entire batchscript, which I've also further improved:

感谢Dave Benham对相关问题的回答,我找到了解决方案!它似乎是WinXP中一个特定的FOR / F错误,猜猜是什么,这里我还在使用WinXP。为了修复主要攻击者,curl-pipe-jq-for-loop,我必须在整个命令之前和之后放置^“。因此整个批处理脚本,我也进一步改进了:

@ECHO off
CLS

:: NPO JSON-extractor geschreven door Reino Wijnsma, 2015 (reino@degeelebosch.nl)

SET batchname=NPO JSON-extractor
SET version=1.1
TITLE %batchname% %version%

SET curl="C:\map with spaces\curl.exe"
SET jq="C:\map with spaces\jq-1.5rc1.exe"

:Check
IF EXIST %curl% (
    IF EXIST %jq% (
        GOTO Input
    ) ELSE (
        ECHO 'jq.exe' niet gevonden.
        ECHO.
        PAUSE
        GOTO :eof
    )
    GOTO Input
) ELSE (
    ECHO 'curl.exe' niet gevonden.
    ECHO.
    PAUSE
    GOTO :eof
)

:Input
ECHO Voer npo.nl programmalink in :
SET url=
SET /P url=
:: http://www.npo.nl/buitenhof/03-05-2015/VPWON_1232766/POMS_VPRO_850040 bijvoorbeeld
IF "%url%"=="" GOTO :eof

SETLOCAL ENABLEDELAYEDEXPANSION
FOR %%A IN ("%url%") DO (
    FOR /F "delims=" %%B IN ('^"%curl% -s http://e.omroep.nl/metadata/aflevering/%%~nxA ^| %jq% -R -r -s ".[1+index(\"(\"): rindex(\"^)\")]"^"') DO (
        ECHO.
        ECHO JSON:
        FOR /F "delims=" %%C IN ('ECHO %%B ^| %jq% .') DO ECHO %%C
        ECHO.
        FOR /F "tokens=1-3" %%C IN ('ECHO %%B ^| %jq% -r "[.tijdsduur,.start,.eind] | @tsv"') DO (
            ECHO Tijdsduur: %%C
            IF NOT "%%D"=="" (
                SET ss=%%D
                SET to=%%E
                SET /A "_ss=((1!ss:~0,2!-100)*3600)+((1!ss:~3,2!-100)*60)+(1!ss:~6,2!-100)"
                SET /A "_to=((1!to:~0,2!-100)*3600)+((1!to:~3,2!-100)*60)+(1!to:~6,2!-100)"
                ECHO Start:     %%D (!_ss!s^)
                ECHO Einde:     %%E (!_to!s^)
            )
        )
    )
)
ECHO.
ENDLOCAL
GOTO Input


Important notes for future reference:

重要说明供将来参考:

jq-syntax:           jq -R -r -s  '.[1+index("("):   rindex(")")]'
cmd-shell:           jq -R -r -s  ".[1+index(\"(\"): rindex(\")\")]"
for-loop:           'jq -R -r -s  ".[1+index(\"(\"): rindex(\"^)\")]"'
for-loop (path): '^"%jq% -R -r -s ".[1+index(\"(\"): rindex(\"^)\")]"^"'

- For the cmd-shell you have to escape the double quotes with a line-feed \.
- While 2 of the closing parenthesis are part of jq's syntax, the 1 between double quotes is not. So when in a for-loop, to prevent this for-loop from closing, you'd have to escape this one with a ^.
- When jq's executable path is put in a variable with double quotes, to circumvent a WinXP bug, you'd then also have to put the entire command between ^", because the parenthesis are now considered special characters! This workaround is compatible with Vista and beyond. (See also DosTips.com)

- 对于cmd-shell,您必须使用换行符\来转义双引号。 - 虽然两个右括号是jq语法的一部分,但双引号之间的1不是。所以当在for循环中,为了防止这个for循环关闭,你必须用^来逃避这个。 - 当jq的可执行路径放在带双引号的变量中时,为了规避WinXP错误,你还必须将整个命令放在^“之间,因为括号现在被认为是特殊字符!这种解决方法与Vista兼容以及。(另见DosTips.com)

#2


Without being able to test it, I suggest to try your batch code as written below:

我无法测试它,我建议您尝试批量代码,如下所示:

@ECHO off
SET "curl=C:\map with spaces\curl.exe"
SET "jq=C:\map with spaces\jq.exe"
ECHO Enter npo.nl program-url :
SET "url="
SET /P "url="
:: http://www.npo.nl/buitenhof/03-05-2015/VPWON_1232766/POMS_VPRO_850040 for example
SETLOCAL ENABLEDELAYEDEXPANSION
FOR /F "tokens=6 delims=/" %%A IN ("%url%") DO (
    FOR /F "delims=" %%B IN ('"%curl%" -s http://e.omroep.nl/metadata/aflevering/%%A') DO (
        FOR /F "delims=" %%C IN ('ECHO %%B ^| "%jq%" -R -r -s ".[1+index(\"^(\"): rindex(\"^)\")]"') DO (
            FOR /F "delims=" %%D IN ('ECHO %%C ^| "%jq%" -r .start') DO SET "ss=%%D"
            FOR /F "delims=" %%D IN ('ECHO %%C ^| "%jq%" -r .eind') DO SET "to=%%D"
            FOR /F "delims=" %%D IN ('ECHO %%C ^| "%jq%" -r .tijdsduur') DO SET "t=%%D"
            ECHO Start:    !ss!
            ECHO End:      !to!
            ECHO Duration: !t!
        )
    )
)
ENDLOCAL
PAUSE

The difference to your code is the position of the double quotes on assigning a string to an environment variable.

与代码的不同之处在于将字符串分配给环境变量时双引号的位置。

The command set is usually used with the parameter variable=value whereby this entire string is the parameter.

命令集通常与参数variable = value一起使用,整个字符串就是参数。

The command

SET "curl=C:\map with spaces\curl.exe"

puts the entire parameter of command set into double quotes. As a result the environment variable curl is defined with string C:\map with spaces\curl.exe without double quotes assigned to it.

将命令集的整个参数放入双引号中。因此,环境变量curl使用字符串C:\ map定义,其中spaces \ curl.exe没有分配双引号。

As an extra bonus of using double quotes on entire parameter string of command set, usually not visible trailing spaces and tabs at end of command line are now ignored.

作为在命令集的整个参数字符串上使用双引号的额外好处,现在忽略命令行末尾的通常不可见的尾随空格和制表符。

But using

SET curl="C:\map with spaces\curl.exe"

results in a completely different behavior. The entire parameter variable=value of command set is now not enclosed in double quotes. As a result of the different position of first double quote in the line the command set creates now the environment variable curl with assigning the string "C:\map with spaces\curl.exe" to it with the double quotes and with including perhaps also existing spaces and tabs at end of the command line.

导致完全不同的行为。整个参数变量=命令集的值现在不包含在双引号中。由于行中第一个双引号的位置不同,命令集现在创建环境变量curl,并使用双引号将字符串“C:\ map with spaces \ curl.exe”赋值给它,并且可能还包括命令行末尾的现有空格和制表符。

The batch code below copied into a batch file and executed demonstrates the differences:

下面的批处理代码复制到批处理文件中并执行,演示了不同之处:

@echo off
set "Var1=String with spaces and "double quotes""   
set Var2="String with spaces and "double quotes""
set Var3="String with spaces and "double quotes" and 3 trailing spaces"   
echo Var1=#%Var1%#
echo Var2=#%Var2%#
echo Var3=#%Var3%#
pause

Character # on output of the 3 variables is used to display where the assigned strings really start and end.

3个变量输出的字符#用于显示指定字符串真正开始和结束的位置。

The output is:

输出是:

Var1=#String with spaces and "double quotes"#
Var2=#"String with spaces and "double quotes""#
Var3=#"String with spaces and "double quotes" and 3 trailing spaces"   #

There are 3 trailing spaces at end of line defining Var1, but they are ignored as the double quotes enclose the entire parameter string of command set.

在定义Var1的行尾有3个尾随空格,但它们被忽略,因为双引号包含命令集的整个参数字符串。

There are no trailing spaces on line defining Var2, but all 4 double quotes are now part of the assigned string and not just the two double quotes around double quotes part of the string.

在定义Var2的行上没有尾随空格,但是所有4个双引号现在都是指定字符串的一部分,而不仅仅是字符串中双引号的两个双引号。

And last the line defining Var3 has also 3 trailing spaces which are also assigned to the variable which is very often not wanted on command lines referencing the value of the environment variable.

最后,定义Var3的行也有3个尾随空格,这些空格也分配给变量,这在引用环境变量值的命令行中通常不需要。

So best is to use always set "variable=value" even if whether variable name nor value contains a space. This notation is simply a protection against invisible trailing spaces or tabs being also assigned to the environment variable.

因此,即使变量名称和值是否包含空格,最好使用始终设置“variable = value”。这种表示法只是一种保护,可防止不可见的尾随空格或制表符也分配给环境变量。

As curl.exe with path containing spaces is now assigned to the environment variable curl without the double quotes, it is necessary to use the double quotes around the complete string containing %curl% which is here simply "%curl%". Same is true for %jq% as being not used inside a longer string and therefore always just "%jq%" can be used.

由于包含空格的路径的curl.exe现在已分配给没有双引号的环境变量curl,因此必须在包含%curl%的完整字符串周围使用双引号,这里只是“%curl%”。对于%jq%也是如此,因为未在较长的字符串中使用,因此始终只能使用“%jq%”。

One last hint: Debugging of batch files is often quite simple. Changing first line from @ECHO off to @ECHO ON or removing/commenting first line results in execution of batch file with showing also what is really executed by command line interpreter. The error in code can be often quickly found now by looking on the command lines processed.

最后一个提示:批处理文件的调试通常非常简单。将第一行从@ECHO更改为@ECHO ON或删除/注释第一行会导致执行批处理文件,同时显示命令行解释器实际执行的内容。现在,通过查看处理的命令行,可以快速找到代码中的错误。


推荐阅读
  • 本文将详细介绍如何在ThinkPHP6框架中实现多数据库的部署,包括读写分离的策略,以及如何通过负载均衡和MySQL同步技术优化数据库性能。 ... [详细]
  • 本文详细介绍了一个既适用于PHP5也适用于PHP7的cURL文件上传解决方案。此示例源于项目需求,旨在通过cURL实现文件上传功能,并解决不同PHP版本间的兼容性问题。 ... [详细]
  • 构建Filebeat-Kafka-Logstash-ElasticSearch-Kibana日志收集体系
    本文介绍了如何使用Filebeat、Kafka、Logstash、ElasticSearch和Kibana构建一个高效、可扩展的日志收集与分析系统。各组件分别承担不同的职责,确保日志数据能够被有效收集、处理、存储及可视化。 ... [详细]
  • This post discusses an issue encountered while using the @name annotation in documentation generation, specifically regarding nested class processing and unexpected output. ... [详细]
  • ElasticSearch 集群监控与优化
    本文详细介绍了如何有效地监控 ElasticSearch 集群,涵盖了关键性能指标、集群健康状况、统计信息以及内存和垃圾回收的监控方法。 ... [详细]
  • 远程过程调用(RPC)是一种允许客户端通过网络请求服务器执行特定功能的技术。它简化了分布式系统的交互,使开发者可以像调用本地函数一样调用远程服务,并获得返回结果。本文将深入探讨RPC的工作原理、发展历程及其在现代技术中的应用。 ... [详细]
  • 本文档汇总了Python编程的基础与高级面试题目,涵盖语言特性、数据结构、算法以及Web开发等多个方面,旨在帮助开发者全面掌握Python核心知识。 ... [详细]
  • Spring Cloud Config 使用 Vault 作为配置存储
    本文探讨了如何在Spring Cloud Config中集成HashiCorp Vault作为配置存储解决方案,基于Spring Cloud Hoxton.RELEASE及Spring Boot 2.2.1.RELEASE版本。文章还提供了详细的配置示例和实践建议。 ... [详细]
  • 微信小程序中实现位置获取的全面指南
    本文详细介绍了如何在微信小程序中实现地理位置的获取,包括通过微信官方API和腾讯地图API两种方式。文中不仅涵盖了必要的准备工作,如申请开发者密钥、下载并配置SDK等,还提供了处理用户授权及位置信息获取的具体代码示例。 ... [详细]
  • 本文探讨了Web开发与游戏开发之间的主要区别,旨在帮助开发者更好地理解两种开发领域的特性和需求。文章基于作者的实际经验和网络资料整理而成。 ... [详细]
  • 本文探讨了如何在Node.js环境中,通过Tor网络使用的SOCKS5代理执行HTTP请求。文中不仅提供了基础的实现方法,还介绍了几种常用的库和工具,帮助开发者解决遇到的问题。 ... [详细]
  • 本文探讨了浏览器的同源策略限制及其对 AJAX 请求的影响,并详细介绍了如何在 Spring Boot 应用中优雅地处理跨域请求,特别是当请求包含自定义 Headers 时的解决方案。 ... [详细]
  • 本文将详细介绍小蚁智能行车记录仪的各项性能,包括其1296P高清分辨率的实际表现及ADAS辅助驾驶系统的功能测试。通过白天、夜晚及低光环境下的视频测试,全面评估这款设备的实用性。 ... [详细]
  • JMeter接口关联与数据提取:正则表达式和JSON Extractor的使用
    在使用JMeter进行接口测试时,常常需要从前一个接口的响应中提取数据并应用于后续请求。本文将详细介绍如何利用正则表达式提取器(Regular Expression Extractor)和JSON Extractor来实现这一需求。 ... [详细]
  • 探讨在使用 Fast-Android-Networking 库时遇到的 addStringBody 方法无法正常工作的问题及其解决方案。 ... [详细]
author-avatar
河南华萃酒业_359
这个家伙很懒,什么也没留下!
PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有